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Abstract. In this note, we give a formula for the Whittaker-Shintani functions 
for the p-adic symplectic groups, which is a generalization of the Zonal spherical 
functions and Whittaker functions. We then use the formula to give an alterna- 
tive proof of a conjecture given by T.Shintani on the unramified calculation of 
L-functions for Sp2 m X GLi. 

1. Introduction 

1.1. Let G be a connected reductive linear algebraic group defined over a number 
field F with its ring of Adeles A, II = <S> V H V an irreducible unitary automorphic 
cuspidal representation of G(A), and r a finite dimensional representation of the 
L-group L G of G. Following Langlands, one may define the partial L-function as 

L s {s,U,r)= l[L( S ,U v ,r v ) (1) 

where S is a finite set of places of F outside of which both G and U v are unrami- 
fied. Langlands conjectured that this partial L-function continues to a meromorphic 
function in C which has only finitely many poles and satisfies a standard functional 
equation relating its value at s to 1 — s. One of the successful approaches to this 
conjecture, the Rankin-Selberg method, is by constructing a global zeta-integral plus 
an Euler product expansion, and equating the unramified local zeta-integrals with 
the "Langlands factors" L(s,U v ,r v ) (referred to as "unramified computation"). 

One of the interesting cases of the above conjecture is the partial tensor L-function, 
where (G, II, r) = (Sp 2n X GL k , n <g> r, "standard".). Here 7r and r are irreducible 
cuspidal automorphic representations of Sp 2n and GL k respectively. The purpose 
of this paper is to give an explicit formula for the Whittkaker-Shintani functions, 
which is one of the key steps towards the unramified computation, for the case 
when 7r is non-generic and k < n, following the global construction of zeta-integral 
in [5]. When ir is generic the unramified calculation is completed in [5] using the 
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Casselman-Shalika formula ([!]) for the Whittaker functions, and the formula for 
Whittkaker-Shintani functions play a parallel role in the non-generic case. 

The main idea of this paper actually comes from [I] for the Casselman-Shalika 
formula for Whittaker functions, and [7] where the Whittaker-Shintani functions 
for orthogonal groups are defined in a similar way and an explicit formula is given. 
However, the calculation in our case is more technical since the Jacobi group we are 
dealing with is not reductive. While the general formula we obtain is not as explicit 
as in the orthogonal case due to its nature, we still have an explicit formula (jSJ) when 
restricted to the torus, which is enough for the unramified computation. 

The author would like to express his sincere appreciation to his adviser, Prof. 
Dihua Jiang, for his support and advice in writing this paper. He also thanks Prof. 
David Ginzburg for his suggestion on the unramified computation. This paper would 
be part of the author's thesis. 

1.2. Let G and M be symplectic groups, defined over a non-archimedean local field, 
of rank n and m respectively with n > m + 1. Let Ind Bo (x) be the unramified 
principle series of G. Let M J be the Jacobi group and B M j its Borel subgroup as 
defined in (jXJ) in Section [2J and let Indg ; (£, ip) be an unramified principle series 

of M J as defined in ([C]) in Section [2j Let U be the unipotent radical of a parabolic 
subgroup p™ - " 1-1 of G and ipu be a character on U which is stablized by M J (see 
(EJ) and (J7D). Then one can define an M J -invariant, (U, ^u)-covariant pairing 
between Indg G (x) and Indg Mj (£, ■0). Let F° and ^ be the normalized spherical 
vectors in IndB G (x) and Indg ^ and we define 



This function is a Whittaker-Shintani function attached to (%, £, ip) (see Definition 
12. 3|) . We will show later that for given (x, £, ip) (unramified) such function is unique 
up to a scalar. Denote by W° ^ ^ the normalized Whittaker-Shintani function which 
is equal to 1 at the identity. In this paper we show the following two theorems (for 
the definition of X°, Z, K M j, p d , A, p f , K G , see Section [2]). 

Theorem 1.1. Let ( X ,0 G C n x C m , and let d e A+, f e A+. Let W°^ be the 

normalized Whittaker-Shintani function attached to (x, Then we have 



W X! ^(<?) = W^(#)F°,F^)- 




(2) 



b(wx,w'{,)d(wx)d'(w'$,)((wx) 



^)(p f )(KO-^)(p d ). 



■wew G ,w'ew u 
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where 



and 



d(x)= n c(xa±x b )U((xi), d'(o= n cfei^n^), 

l<a<b<n i=l l<a<b<m j=l 

Mx,o= n c\xi-tj+h- n c'i-xi+ti+h 

i<j+n—m i>j+n—m 

II c^+i) n r 1 (x l +e, + ^) 

l^i^" 1 l<i<n 
l<j'<m 

Theorem 1.2. Under the same notation and assumption as in the previous theorem, 
the support o/W°^ z's on 

|J ZUK M ,(p d Ap f )K G . (3) 

dgA+,feA+ 

If we let £(d',f) = J x o daW°£ ^(p d 'a:p f '), £/ien t/iere exz'sts a(d') > independent 
of (x, £, i>) such that 

W^(p d Ap f ) = £ a(d')£(d', f + d - d'). (4) 
d' 

where d' runs over the set {d' | d' GA+ , f + d — d' e A+, d' < d} and a(d) > 0. In 

particular, we have 

W°^(p f ) = £(0,f) (5) 

The paper is organized as follows. In Section [2] we give the notation we use in 
this paper. In Section [3] we use the Rankin-Selberg convolution to find an integral 
expression of the pairing when (x, £ ) belongs to Z c C C n x C m which contains 
a Hausdoff open set. In Section [4]we show that the pairing l x ^^ between IndB G (x) 
and Indg J (£, ip) satisfying Condition A (see Definition 12. 21) is unique up to a scalar. 

Mr 

Then in Section Owe apply the Bernstein's theorem to extend this pairing defined 
by the integral to generic (x, £). m Section Owe discuss the double cosets of G 
on which the Whittaker-Shintani function is supported. By considering the vectors 
invariant under certain open compact subgroups (in Section [7]) and applying the 
intertwining operators (in Section [8]) we give an explicit formula in Section [9] for 
the Whittaker-Shintani function attached to generic (%,£), and we obtain its value 
at the identity by an combinatorial argument in Section 1101 After showing the 
uniqueness of the normailized Whittaker-Shintani function in Section lll[ we apply 
the Bernstein' theorem again to extend the formula to all (x, £) in Section 1121 In 
Section 1131 we use the formula we obtained to give an alternative proof of in [HJ 
Theorem 6.1], the unramified calculation of L-fucntions for Sp 2n x GLq. 
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The application in the last section is in fact a special case of Theorem 4.3], the 
unramified calculation of L-functions for Sp2 n x GL^. The proof for the general case 
will be shown in our [TU|. We also expect parellel results for unitary groups (with 
respect to skew hermitian forms). This will be covered in the future. 

2. Notation 

In this paper, we let F be a non-archimedean local field of characteristic 0. Let 
O be its maximal compact subring and p the uniformizer. Suppose the order of 
the residue field is q which is not a power of 2. All the groups are defined over F. 
Through out the paper we fix if; to be an additive character on F with conductor 0. 

(A) Groups. Let G = Sp 2n , H = Sp 2m+2 and M = Sp 2m , where m, n are two positive 
integers with n > m + 1. M (or H) embedds to G as diag(l n - m , g, l n - m ) (or 
diag(l n - m -i,g, l n _ m _i)) for g £ Mjor g £ H). Let M 2n x2n{F) be the 2n by 2n 
matrix over F. For any subgroup G of G, and any i > 0, we define G* as 

G* = G n (/ 2n + M 

2n x In 

Let Kq = G°, and K M = M PI K G . Let J be the Heisenberg group of dimension 
2m + 1 embedding to H as 

(I x y z \ 
1 % 
1 — t x 

V i J 

where x, y E F m , z £ F. Let M J = M x J, and K m j = K M x J°. We let 
X(x) = J(x,0,0), Y(y) = J(0,y,0), Z(z) = J(0, 0, z) and let X,Y,Z be the 
group of them respectively. Let Bq, Bh and Bm be the standard Borel subgroup 
of G, H, M, and B M j = B M x (Y x Z), and let N G , N H , N M , N M j be their unipotent 
radical respectively. Let Tq be the toral part of of Bq, and let 

A+ = {(di, 4) e Z k | d x > d 2 > ... > 4 > 0} 

T+ = {diagih, ...,t n ,t~\ ...,t^) | |*i | < ... < \t n \ < 1} 

T G = {t" 1 | t £ T+} 

The definition of T^J and are similar. Let p^-™ -1 be the standard parabolic 
subgroup of G with Levi decomposition 

pn-m-l = Q L n-m-l x H X U. (6) 

Let ipy be the character on U given by 

n—m—l 

^u(«) = V»( Yl U i,i+l)i ( 7 ) 



J(x,y,z) 



i=i 
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which is stablized by M J . We denote by 1q, Im the Iwahori subgroups of G and 
M, and Im = Im * J°- Let Wg, Wm be the Weyl group of G and M with respect 
to Tg and Tm- 

(B) Elements. Let be the longest Weyl element in G. For k < n, and given 
h,...,t k e_F*, we let d k (h, t k ) = diag(I n _ k , t u . . . , t k , t^ 1 , . . . , t^ 1 , I n _ k ) e 
Tq. Let Z = Z U {oo}, and let v be the normalized valuation from F to Z. 

For a, b G Z , we define an order in Z such that a > b if and only if a — b G 
(N U {oo})*. We define min(a, b) = (min(ai, 61), . . . , min(afe, b k )). When a G 
Z™, we let A(a) = X(p ai , . . . , p am ). Here p°° = 0. Let A = A(0). For a = 
K a k ) G Z fc , we let p a = 4(p a \ p afc )- 

(C) Representations. Let \ an d £ be unramified characters on Tg and Tm- We 
parametrize them as \ — {Xu ■■■iXn) £ C n and £ = (£1, ...,£ m ) G C m such that 
Xidnih, ...,t n )) = nr=i N Xl and f (d m (*i, -,0) = IVJLi Let Ind^,^) 
be a representation of M J consistiting of smooth functions on M J such that 

/(MO, V, z)m 3 ) = £6* (b m )i;(z)f(m 3 ), 

M J 

with M J acting by right translation. Sometimes we write £ip as a character on 
B M j such that 

&{b M J(0,y,z)) = £{b M )i;{z). 

Remark 2.1. Although B M j\M J is not compact, functions in Indg are com- 

pactly supported on B M j\M J by smoothness. In fact by Iwasawa decomposition on 
M, we have 

M J = B M .,XK M . 

Suppose f G Indg J } (£, ip) which is rightKf invariant for some open compact subgroup 
Kf. Note that Kj\K m j is finite. Let k be a representative in one of the cosets and 
suppose f(xk) 7^ 0. Then by the smoothness of f , f(xk) = f(xyk) when y is in a 
neighbourhood of G F m . But note that f(xyk) = xjj(2(x,y))f(xk), so ij}((x,y)) = 1 
for all such y, which implies that x belongs to a compact subset. 

In particular, if f is K M j -invariant, then f(x) 7^ implies x G X°. So the spherical 
vector in Indg ^ is supported on B m jK m j, and is unique up to a scalar. 

(4) Functions and Functional. We denote by £(s) the local zeta function 
as C(s) = (1 — q~ s )~ l ■ For any set X we denote by Chx its characteristic 
function. For (p 1 G C£°(G), we let 

F x M(g)=[ x'^M'piM^q- (8) 
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Then the map (pi i— > F x ((pi) is surjective from C£°(G) to Ind Bc (x). Similarly 
for ip 2 G C~(M J ), we let 

F^(</? 2 )(m J ) = I (M^2(&mJ™ J M&mJ- (9) 

The map ip 2 (->■ F^(y? 2 ) is surjective from C£°(M J ) to Indg '0). Let 
]C X £^ be a function defined on G such that 

/c x ,^(6 G ^ G Aj(o,y^)6 M w) = ^^gKHm^)^ 1 ^), (io) 

and K x ^{g) = for all other g. For ^ G C C °°(G) and y> 2 e C^°(M J ), let 

W.i^i'^Xff) = [ dg' [ dmViij'l^^fg'f^mt 1 )^^), (11) 

and let I^Q?) = I XiW (C7i Kg , Ch Km3 ){g). Let F° = F x (C/i Kq ) and F^ = 
F^^(C/iK M j) be spherical elements in Indg G (x) and Indg Mj (^, 0) respectively. 
Let %q be the spherical Hecke algebra of G, and Hmj,^ be the spherical Hecke 
algebra of M J with respect to if) as defined in section 4 of [5], and let them act 
on Inde G (x) Kg and I n dB Mj (£; ip) Kyi1 by characters lo x and oj^ respectively. For 
any function / on G, let (L(g )f)(g) = fig^g), and (R(g Q )f)(g) = f(gg )- 

Definition 2.2. A pairing l x ,£,ti> between Indg G (x) and Indg^ (£, ip) is called 
satisfying Condition A if 
(i) l x ^(F x ,F^) = l x ^(R(m J )F x , R(m J )F u ,) for any m J G M J . 
(H) l x ^(R(u)F x ,F^) = 'ip l j(u)l x ^(F x ,F^) for any w G U. 

Definition 2.3. For ( X ,0 e C n x C m , a function W x ^ G C°°(G) is called 
a Whittaker- Shintani Function attached to (x,£) if 
(*) W X £ : ip(zuk M jgk G ) =ip- 1 (z)i/j v (u)W x ^(g). 

(ii) L(Lp M 3)R(Lp G )W X £^ = U)z((p M 3)u; x ((p G )-W Xt M for any ip w G H m j,^ and 
<Pg e Kg- 

The space of Whittaker- Shintani functions attached to (x, £, ip) is denoted by 
VV«S X £ Sometimes we omit ip because it is fixed in this paper. A Whittaker- 
Shintani function is called a Normalized Whittaker- Shintani function if it 
equals 1 at the identity. 

3. Integral expression for the pairing 

We first use the function JC X ^^, as defined in (JTUJ), to construct a pairing between 
Indg (x) and Indg T (£,0) satisfying Condition A. For any element g G Bgu^Ng, 
the way to express g = bw^n with b G Bq and n G Nq is unique. From this it is 
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not hard to see that the set Bqw^AYZBmU has the same property. So the function 
1C X £^ is well-defined. Moreover B G w^AYZB M U is zarisky open in G by lemma 03. 3D 
below. Let y x e C^°(G) and <p 2 e C^(M J ), and let F x (( / ? 1 ) and F^(<p 2 ) be defined 
as in (jHD and (jUJ). Then we let 

£( F xM)(9)= , F x(Vi)W C x,€,^(^) d ^> 

• / Bg\G 

where dg is the right G-invariant functional on Ind Bc ((5g G ) determined by the Haar 
measure of G. By direct calculation we have 

£(f x Oi))0) = / M9'g)£ x ,u>(9 f )dg , 

J G 

which is convergent when /C Xi ^^ is continuous on G. £(F x (<pi)) satisfies 

(1) Sim^FxMfo) = fa(u)£(VxM)(9) when u e u - 

(2) When restricted to M J , £(F x (</?i)) e Ind^^" 1 , 

So £ actually gives an M J -homomorphism from the twisted Jacquet-module 
^Indg G (x)) u ^ to bidg^ ip). Now we consider the integral 

W(F X (Vi),F^W)= / ^ i ^(F x (^ 1 ))(m J )F^(^ 2 )(m J )dm J 

where dm J is the right M J -invariant functional on Indp (8-£ ) determined by the 
Haar measure of M J . Substituting £ and by definition, we have 

WFx(¥>i)> F e,*M)= / / ^(^^(^(mt^sKld^ds' (12) 

JG iM J 

Note that the right hand side is actually I X £^({p 1 ,(p 2 )(e) as defined in ffTTj) . 

It is easy to see that the pairing l x> ^(F x (<fi), F^^(<f 2 )) satisfies Condition A if the 
integral is convergent, and the integral is convergent if JC x ,£,ii> is continuous on G. In 
the rest of this section we will prove the following proposition. 

Propostion 3.1. Let Z c be the set of unramified characters (x,0 satisfying 

R e {Xi ~~ Xi+i) — 1 f or 1 < 2 < n — m — 1 

_ Re(xn-m-i+j ~ £j) >\forl<j<m 
Re(-Xn- m +j + £j) > \ forl<j<m 
Re(Xn) > 1 

then when (x,£) G Z c , the function JC X ^ is continuous on G, and as a consequence, 
the integral (TjJ|) is convergent. 
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Since KL X £^ is defined continuously on an Zariski open subset of G (we will see 
this soon) and extend by to G, we only need to show the continuity outside the 
Zariski open set, for the function \K X ^^\. The method we use here is similar to that 
in0. 

First by the Bruhat decomposition we have 

G= |J BgwNq. 

And we know that B G -u; G N G is zariski open in G. In fact we have 

Lemma 3.2. There exists a k £ o[G] for 1 < k < n such that 

B G w; G N G = {g | a k (g) ^ for allk}. 

Proof. For g £ G, let its matrix be g = (gij)i<ij< 2n - Let A^n = {1, 2, . . . , 2n}. 
For / = (ix,...,i k ) and J = {j\,...,j k ) both belonging to J\f£ n , we Let g u = 
(9is,jt)i<s,t<k- We define 

A IJ (g)=detg IJ . (14) 

For 1 < k < n, let I k = {2n + 1 - k, 2n + 1 - (k - 1), . . . ,2n}, and J k = {1,2,..., k}, 
and we take 

a k (g) = A Ik>Jk (g). (15) 



Then one can check that 



(1) For any n 1 ,n 2 £ N G , «fc(ni5fn 2 ) = a k (g). 

(2) a k {d n {ti, t n )gd n {si, s n )) = Ui=i " ot k {g). 

(3) Let w £ W G . If a k {w) ^ for all 1 < k < n, then w = wfi. 

Combining these properties with the Bruhat decomposition of G, we have our lemma. 

□ 

Next we have 

Lemma 3.3. There exists Pi £ o[G] for 1 < I < m such that 

B G u> G AYZB M U = {g £ G | a k (g) ± 0, p t (g) ^ for all 1 < k < n, 1 < I < m}, 
where a k is as defined in lemma 

Proof. Note that for any w £ W, B G wN G = B G u;XUN M j. For any X(xi, . . . ,x m ) £ 
X, we have X(sei, . . . , x m ) = s _1 X(ri, . . . , r m )s, where s = d m (si, . . . , s m ) £ Tm such 
that 

, if Xi ^ 0; 




THE WHITTAKER-SHINTANI FUNCTIONS FOR SYMPLECTIC GROUPS 



9 



From this we can see that 

B G wN G = |J B G u>X(r)B MJ U, (16) 

re{0,l} m 

and when w = Wq, the union is disjoint. For 1 < I < m, we let 

J' l ={l,2,...,(n-m),...,n-m + l}, 

and we define 

Pl(9) = A-In-m+l-uJ'tid)- (17) 

Then (3i satisfies 

(1) fii{nig) = (3i(g) for any n\ G Nq. 

(2) (3i(gn 2 u) = (3i(g) for any n 2 G N M j and u G U. 

(3) AK(ti, . . . , t n ) g d m { Sl , Sm )) = n?=T t- 1 ■ n;- 1 ! t~ l _ m+j ■ nj=i ^ • aq?)- 

(4) (3i(w^X(r)) = ±r^. The sign in front of r; depends on I, which is not impotant 
since we are only interested in |/^|. 

In fact, (1) is by the definition of Ik while (3) and (4) are by direct calculation. For 
(2), note that if we only consider the first n column of {gij), multiplying elements in 
N m jU from the right corresponds to column operations adding multiples of column 
h\ to column k 2 where 1 < ki < k 2 < n with k\ ^ n — m. On the other hand elements 
in J[ are consecutive from 1 to n — m + / with n — m missing, so Aj n _ m+l _ lt ji(g) is 
invariant under such column operations. 

So for g G B G u;X(r)B M jU, ctk(g) ^ and Pi(g) ^ for all 1 < k < n and 
1 < I < m if and only if w = (by lemma l3T2|) and X(r) = A (by the property of 
A's), completing our proof. □ 

Remark 3.4. If we let Wi = e\ + . . . + e« G Hohi(Tg, GLi) and Wj = e[ + ...+ e'j G 
Hom(TM, GLi) be the dominant weights of G and M with respect to Bq and Bm ; 
then the properties of and Pi actually shows that under the Bg x B m .t action, 
at has the highest weight (wk,0) when 1 < k < n — m, and {vjk,vj' k _, n _ m \) when 
n — m + 1 < k < n, and fii has the highest weight (zo n _ m+ i_i, w[). 

Now we can expressed JC X ^^ by and First we have 

Lemma 3.5. Let g = d n {t\, . . . , t n )nGWQ \d m (si, . . . , s m )n M ju G Bgu>^AB m jU ; we 
have 

|ai(flO _1 | */* = 1 ; 

\cti-ict[ l (g)\ if 2 < i < n — m; 

(^-(n-m)^" 1 ^)! ifi>n-m 



and 



\ s j\ = \Pja n - m+j -i(g)\ forl<j <m 
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By this we have 
Lemma 3.6. For g £ BqiOq AB^jU, we have 

n—m—l m 

l^X&Ml = II \(XiX7+l\ ■ T 1 )^^))! ■ II KXn-m-l+jt^l ■ \~H a n-m+j(g))\ 
i=l j=l 
m 

■ Kxni • n(a n ( 9 ))\ ■ n • rhmm 

The proof of these are by direct calculation. Note that a^, (3i are continuous 
functions on G, so when the assumptions in proposition ^. H are satisfied, the extension 
of \JC X) ^(g)\ by to outside the set B g u>^AB m jU is continuous, and so JC x ^^(g) is 
continuous. 

4. Uniqueness for the pairing for generic (%,£) 
The pairing l X) ^ satisfying Condition A corresponds to the homomorphism 

Hom BMj u(Ind« G (x),rV- 1 4 Mj ®^u)- (18) 
In this section we prove that 
Propostion 4.1. For generic (x, £), 

dimHoniB^ulIndijx)^- 1 ^- 1 ^ ®^u) < 1. 

Let lid be the union of double cosets of Bg\G/B m jU with codimension < d. Then 
Uq = Bgu^AB m jU is open in G, and for any d > 1 we have the exact sequence 

S(^_!) S(W d ) -)> £ S(W) -)• 0. 

codim W=d 

Obviously we have 

dimHoniB^uand^Jx,^)^" 1 ^" 1 ^ ®^u) < 1, 
so we only need to show that 

Lemma 4.2. Suppose (x,Q is generic, and let U = Bg(?B m ,jIJ be a double coset 
different from Uq, then 

dimHomB^uandgjx,^)^" 1 ^" 1 ^ ®^u) =0. 

Proof. When U = B G gB M .iU, we let G g = B m jU fl g~ l B G g, then 

Hom BMJ u(Ind« G (x,ZY),e-V' 1 4 Mj ® ^u) 
=Hom G9 (( 7 ^ 1 (x4 G ) ® £#b1 ® ^>u\ <5 9 ) 
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where 8 g is the modulus character of G g . So we need to show that 

rti)-^'^ 1 '^ 1 ^ 1 (19) 

when restricted to G g for any generic (x>0- R- eca U from (TIB]) that 

G= |J B GW X(r)B MJ . 

uiGW G ,rG{0,l} m 

So we can assume g = uX(r) with either w 7^ or r 7^ (1, 1, ... , 1). What we need 
to find is b\ E Bq and b 2 £ B M j U such that 

6i5 = 9h 

1 _i (20) 

First suppose (1, 1, ... , 1). In this case we claim that Tg PI (^Tm^" 1 ) contains 
a nontrivial torus. Let t = d m (ti, . . .t m ) E Tm. Note that Tg is stablized by the 
adjoint action of Wg, so it suffices to show that there exists a nontrivial torus T s of 
T M such that when t G T a , X(r)" 1 tX(r) G T G . Note that 

X(r)- 1 tX(r) = t • X((l - ti)n, (1 - t 2 )r 2 , . . . , (1 - t m )r m ), 

j-th 

so when Tj = for some j, we can let T s = {t = d m (l, . . . , t , . . . , 1)} be the 
torus we claimed. Then since (%, £) is generic, one can find some b 2 G T s and 
b\ = gb 2 g v G T M so that (I2D1) is satisfied, completing the proof for this case. 

Now suppose r = (1, . . . , 1) G F m and w 7^ Wq, so X(r) = A by our notation. In 
this case there is a simple root a in G such that wN a w~ l G Nq. 

When a = e, — e»+i with l<i<n — m — 1, note that A G M J stablizes ipu, so 
■?/'u(A _1 n Q ,(t)A) = ip\j{. n a(t)) 7^ 1 for some t E F. On the other hand, wn a (t)w~ l E 
Nq by our assumption. So let b\ = wn a {t)w~ l and b 2 = A _1 n a (t)A we have (I2D1) . 

i'-th 

When a = ej — e i+ i with i > n — m, we let r(i, i) = X(l, . . . , 1 , 1 + t, . . . , 1), 
where %' — % — (n — m). Then for t / -1 we have 

wn a (t)\ = wX(r(t,i))n a (t) 

= (c/" 1 (r(t, i)) ) w w\d m (r(t, i))n a {t). 

So let bi = (d^(r(t,i))n a (t)) w and b 2 = d m (r(t,i))n a (t). For generic (x, £), we can 
always find some t E F so that ((2D]) is satisfied. 
When a = 2e n , we have 

wn a (t)X = w\Y(t)Z(-t)n a (t). 

1 

So let 61 = (n a (t)) w and 62 — Y(i)Z(— t)n a (t), we have (x^I G )(^i) = •"•> an< ^ we can 
find some t E F so that "0(Z(— i)) 7^ 1, so (120]) is satisfied. □ 
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5. Rationality(I) 

In this section we show that the pairing l Xl ^ defined in (|T2|) can be extended to 
all generic (x, £). 



Lemma 5.1. Given (pi and(p2, the pairing l x ^^(F x ((pi), F^f^)) as defined in [Ft 
can be extended as a rational function on (%, £), and for generic (x, it is the pairing 
between Indg G (x) and Indg "0) satisfying Condition A. 

Proof. By lemma f!8.4p . which does not depend on the rationality of l Xl ^, we know 
that for (x, e £ c , 

Z x ,^(F x (Ch lG ), F^(Ch lM )) = Vol(I G )Vol(I M ). (21) 

Now we apply the Bernstein theorem. Note that by the last section, the pairing l x ^ 
satisfying Condition A and (l2Tj) is unique for generic (xO- When (x, £) G £ c , the 
pairing / Xi ^^, defined by 

i x^( F x(¥'i)> F «,V'(¥'2)) = W^C^IjVz) 
satisfies these conditions. So it extends to a rational function in (x, £) when tpi 
and </9 2 are given. Moreover, it is regular at generic (x, £,) where the uniqueness is 
valid. □ 

From now on we still denote by l x ,^ its own meromorphic continuation to all 
generic (%,£), and similarly for l x ,^(g) and l x> ^(g)- 

6. Double coset decomposition 

Let W Xj £ )? /,(g) be a Whittaker-Shintani function. In this section we discuss the 
support of W X £^(g) on the double cosets ZUK M j\G/K G . We denote by gi g 2 when 
gi and g 2 belong to the same double coset. We are going to show that 

Theorem 6.1. The support ofW x ^ :1 p(g) is contained in the double coset 

|J ZUK MJ (p d Ap f )K G . 

deA+,feA+ 

First, by the Iwasawa decomposition on G and the Cartan decomposition on M, 
we have 

G = ZUK MJ (XYTr m T+)K G . 

Here T?"™ is the embedding of GL"" m to G as 

(ii; •••) t n _ m ) d n (ti, t n _ m , 1, 1, . . . , 1). 

So we only need to consider the support of W Xj ^^ on the set XYp( a;b ) where a G Z n_m 
and b G A+ We have 
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Lemma 6.2. Suppose W x ^ t ^(xyp^' h ') ^ 0. Then y £ Y° and a £ A n _ m 
Proof. The proof is given in lemma 2.1 in [9]. First for any z £ Z°, we have 
W\. £ .,(.r//p' al ") = W^^^xyp^z) = ^(p 2o *-»z)W x ,^(xyp( a ' b )). 
So a n _ m > 0. 

To show y £ Y°, we argue by contradiction. Let y = Y(yi, y m ) with |^-| = |p|~ r , 
r > 0. We let E a be the root subgroup in G of the root a. Let E a (t) be the canonical 
embedding of F to E a . Define EUt) similarly on M. Then for any t £ O*, 

W x . i ,(.r//p ;, - ,,; ) = W x , 5 ^(xyp( a ' b )^ 2e3 (p^ + ^)) 
=W x ^(xyE>_ 2ej (p r tW a ») = ^p r ty^)W x ^(xyp^) 

But note that p r ty 2 j £ u>~ r 0*, one can choose t £ C* so that ip(p r tyj) ^ 1, which is 
a contradiction. So y £ Y°. 

To show a £ A+_ m , we also argue by contradiction. Since we already have a n _ m > 
0, we assume that a» < aj + i for some i < n — m — 1. Note that when z < n — m — 1, 
E ei - ei+1 £ U. For any t £ (9, we have 

W x ^(xyp^) = W x ^(xyp^E ni (t)) 
=i;(p a ^H)W x ^(xyp^) 

But when a,j < Oj+i, we can always find some t £ O such that ^(p a " _0i+1 i) ^ 1, 
contradicting the assumption. So we have a £ A+_ m . 

□ 

By this lemma the support of W X) £^ is on ZUK M j(XT+_ m T+)K G . Next we narrow 
the support further on the X-part. Note that if v{xj) = q for 1 < % < m, then 
X((xi, . . . , x m ))t A(c)i for any t £ T G . So we only need to consider for which c that 
A(c)T+_ m T+ is contained in the support. 

Lemma 6.3. Suppose x = A(c), then for any t £ T n; 

xt \(min(c, 0))t 

Proof. Note that A £ K M j, so x t Xx t £ T^A(min(c, 0))tT^. □ 

By this lemma, we just need to consider the support of W x ^^ on xp( a;b ) with 
x = A(— d) for some d > 0. 

Lemma 6.4. Let x = A(— d) with d > 0. Let a £ A+_ m and b £ A+. Suppose 
W x>SiV ,(xp( a ' b )) ^ 0, then d < b. 
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Proof. Suppose not, so we let dj > bj for some j. Let t G O*, then 

W x ^(xp^) = W x> ^(xp^E' 2e .(t)) = W x ^(xE' 2ej (p 2b n)p^) 
=^(p%- 2 <^)W x ,^(Y(0, . . . , 0, pj^, 0, . . . , 0)xp( a ' b >) 

3-th 

By lemma f!6.2p . when W Xi? ^(xp^ a ' b ^) 7^ 0, we have p 2b i~ d i G C, so the last formula 
equals ^fj(p 2bj ~ 2dj t)W x ^^(xp^ a ' h ^) . But when dj > bj we can choose some t E O* such 
that ^(p 26j_2d ?t) 7^ 1, contradicting the assumption. □ 

Note that when x = A(-d) with d > 0, xp (a,b) = p d Ap (a,b ~ d) . Let r = b - d. So 
by the previous lemmas, the support of W x ^^ is on the union of ZUK M j(p d Ap < ^ a ' r ' ) )KG 
for all a G A+_ m , d > 0, r > and d + r G A+ . The following two lemmas help us 
to narrow our choice of a, b, d so that we get theorem (16. ip . We also need to use 
them in the later calculations for the Whittaker-Shintani function. 

Lemma 6.5. Suppose g = p d Ap^ a ' r ^ with a G A+_ m; d > ; r > ; and d + r G A+ , 

then 

(1) Suppose a n _ m < r 1 . Let f = (a n _ m ,r 2 , . . . ,r m ). Then 

p d Ap (a,r) p d+r-r Ap (a,r) 

Moreover, if d G A+ , then so is d + r — f . We call the process from (d; a, r) 
to (d + r — r; a, r) Operation 1. 

(2) Fix i, let r = (r ly ...,f m ) where 

! r i if j > i an d r i > r i 
Tj otherwise, 

then 

p d Ap (a,r) p d+r-f Ap (a,f) 

Moreover, if a n - m > r\, then a n _ m > f\; if d G A+, then so is d + r — r. 
For given i, we call the process from (d; a, r) to (d + r — f ; a, f ) Operation 

(3) Giveni, let d = (d±, d m ) where 



di 



di if j < i and dj < d, L 

Lj = \ 

I dj otherwise, 
then 

W x ,^(p d Ap^) = W x ,^(p d Ap( a < r+d - a )). 

Moreover, if (a, r) G A+, then so is (a, r + d — d). For given i, we call the 
process from (d; a, r) to (d; a, r + d — d) Operation (3,i) 
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Proof. For part 1, note that when a < r 1; 

p d Ap (a,r) p d Ap (a,r) A 

p d A((a„_ m - n, 0, . . . , 0))p^ p d+r - F Ap( a ' F ), 
so we have part 1. For part 2, we fix i, then we have 

j \j>i,rj>ri 

I] ^_ £j (p di+r - d ^ r J)p d A(f - r)p^ p d+ - f Ap^ 

j \j>i,rj>ri 

the rest of part 2 is correct since d + r £ A+. Part 3 is similar to part 2. For fix i, 
we have 

j\j<i,dj<di 

p d A(d - d)p (a - r) J] £;_ ej (p d J +r j- di - ri ) p d Ap (a ' r+d - d) . 

j\j<i,dj<di 

The rest of part 3 is correct since d + r £ and f\ <r±. □ 

Propostion 6.6. Suppose a £ A+_ m , d > 0, r > and d + r £ A+ . Starting from 
(d; a, r), if we take the Operation 1, and then Operation (2,i), fori from 1 to m, 
and then Opeartion (3,i), for i from m to 1, we will end the process with (d; a, r) 
such that 

(1) d + f = d + r. 

(2) d > d, so equivalently , r < r. 

(3) d £ A+ , and (a,r) £ A+. 

(4) For any triple (d';a, r') satisfying (l),(2) and (3) above, d < d'. 

In other words, for (d; a, r) such that a £ A+_ m , d > 0, r > and d + r £ A+ , 
there exists (d;a,r) satisfying (1),(2) and (3) above such that p d Ap( a,r ) p d Ap( a,r ). 
Moreover, among all the triples satisfying (1), (2) and (3), (d;a, f) is the one with 
the smallest d. 

Proof. Part (1) is obvious. Part (2) is true because in all the operations, either d 
increases or r decreases, and by (1) they are equivalent. Part (3) is true because by 
opeartion 1, a n _ m > r ly and after operation (2,i), > Tj for all j > i, and after 
operation (3,i), d{ < dj for all j < i. 

To prove part (4), we let (d'; a, r') satisfy (1),(2) and (3). Note that (d; a, r) is the 
result of 2i + 1 operations on (d; a, r). We show below that for any triple (D; a, R) 
satisfying D + R = d' + r' and D < d', it still satisfies the same conditions after one 
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of the operations in lemma (16. 5p . Repeating it for 2i + 1 times we prove (4) since 
initially we have d < d'. 

To be precise, let (D; a, R) be such a triple, and suppose after one of the opera- 
tions in lemma (16.51) it becomes (D'; a, R'). We need to show D' + R' = d' + r' and 
D' < d'. The former one is obvious by the definition of the operations. For the lat- 
ter first suppose the operation we took is (3,i), then for any j, either Dj = Dj < dj, 
or Dj = Di, in which case j < i, implying Dj = Di < d- < dj. So Dj < dj anyway 
and hence D' < d'. If the operation we took is (1) or (2,i), we can similarly prove 
that R' > r', which implies, by D' + R' = d' + r', that D' < d'. So in any case 
D' < d'. □ 

Following lemma (16.51) and (16. 6p . theorem (16.11) is implied. 

7. Vectors invariant under certain open compact subgroups 
7.1. The Ic-invariant vectors in Indg G (x)- 

Let Ig be the Iwahori subgroup of G. For w G Wq, let <&™ x be the element in 
IndB G (u>x) defined as 

<pr x (g) = [ d t b Ch^bg^wxr^b). 

JBq 

Then <£\ x is Ic-invariant. And we have 

<z>r x (i) = Voi(B G ni G ) = i. 

For a Weyl element w and a character \ on the Tq, let T^ be the Intertwining 
operator from Indg G (x) to Indg G (wx) defined as 



T w , x f(g) = dn f(w 1 ng). 

JNniuNii)- 1 



The integral is convergent when Re(x) is sufficiently large, and by [3] it continues 
holomorphically to generic x- We write T w ,\ as T w when there is no risk of confusion. 
For generic Xi the G-intertwining operator from Indg G (x) to Indg G (u>x) is unique 
for every w G Wg- So 



T^j o T W2 c • r ^w\W2 



for some constant c. Moreover, if we let 

C«x,«» 



Caix) " C((x,«) + i) 

c w(x) = n c «(x)> 

a>0,wa<0 
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then by theorem 3.1 in [2] 



T F°=r (y)F° 
So if we let 

Tu),x ^iu(x) ^-WjXi 

then 

T T = T 

Following section 5 in [1] we state the following results without proof. 

Lemma 7.1. Elements in {T w -i<Pi X } w£Wg form a basis of lnd^ G (xY G ■ 

By this lemma, we have 

Lemma 7.2. one can express F° as a linear combination of {T w -i@i X } w ew G - ^ n 
fact we have 

F^VoKIg)" 1 £ c WQ {wxfT w -^i X - 

wGWq 

Next we have 
Propostion 7.3. For a £ Tq, 

R{Ch IaalG )$i = Vol(I G ) X H(a)<2>* (22) 

So as a consequence, 

R(Ch lGalG )F° x = Yl c W0 (wx)(w X )S-Ha)T w -i^ x - (23) 

w£Wq 

7.2. The T M -Invariant vectors in Indg j(C)V')- 

A similar discussion can be applied to M J with some modification. Let Im be the 
Iwahori subgroup of M, and let Im = Im x J°- Consider the space 



*< mJ (^) Im 



We have the following lemma 
Lemma 7.4 

Proof. Since 



Lemma 7.4. Any f £ Ind B .(£, V0 Im zs supported on B m jW m Im- 



M = BmWmIm, 

we have 

M J = B mJ XW m Im 
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Suppose f G Inde t (^,^) Im with f(X( x)w) 7^ 0, then since Wm normalizes J°, we 
have 

/(X(a» = f{X(x)Y(y)w) 

for any y G O m . But then 

f(X(x)Y(y)w) 
=f(Y(y)X(x)Z(2{x,y))w) 
=1>((x, y ))f(X(x)w). 

So i/j((x,y)) = 1 for any y G O m , which means x G O m , completing our proof since 
B m jX°W m Im = B m jW m Im- □ 

By this lemma, we have 

dimIndB^(£,^) lM < Card(W M ). 

For any character £ on Tm, let be the intertwining operator from Indg J (£, ijj) 
to Indg J J (w£, ip) defined as 

T&*(/)(0)= I 7 n f{nr x ng)6n. 

JN J n«>N J u>- 1 \N J 

Similar to the previous subsection, the integral is convergent when Re(£) is suffi- 
ciently large, and continues holomorphically to generic £. In fact the only difference 
is that we are integrating on part of X, on which smooth elements in Indg } (£, ip) is 
compactly supported. For generic £, the M J -intertwining operator from Ind^ 3 (£, ip) 
to Indft 1 (w^ip) is unique, so we have 

for some constant c. By a similar method as in theorem 3.1 in [2] we have the 
following result. 

Lemma 7.5. For a being a simple root in M, if we let 

and let 

Cw(0 = n 

a>0,ioo!<0 

T^(F^)(e) = c w (0 
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So if we define 
then 
Let 

then we have 



= F^(Ch, 



yjftrf(a) = J ^ 5 b mJ (6) if g = bn x, b G B^j, n G N M ' , x G X°. 
\o ' if gi B^N^X 

Lemma 7.6. T/ie set {T u -i\Pi } w6 jy M forms a basis 0/Ind.B J J (£, ^>) Im 

Proof. Since dimIndB J T (e,^) lM < Card(W M ), it suffices to prove that elements in 
{T„-i$i }™ew M are linear indepedent. Consider 

By the definition of \pf , the integral is non-zero implies 

ur 1 ]^^ n b mj n^ m x° ^ 0, 

which implies 

w - 1 N J nB MJ N^ M x°K M )- 1 ^0- 

Note that w^N 3 C BmC 1 ^ k J, and B^N^X ^) -1 C B m (^o I )~ 1n m x J, so 
by the Bruhat-decomposition on M we have 



if T w {^){wf) ^ 0. On the other hand, when w = 

%™(^)«) = (^o m ^))"7n-x dndx ^( nx ) 

which equals 

(c % m(0)- 1 Vo1(I m ) 

by the definition of \pf'^. Now let T w m w acts on all elements in {T w -i fi^} weW - M 
and evaluate them at w^f. Only T w m w o T^-i/i * is non-zero. So they are linear 
independent, and hence form a basis of Indg i (£, i/jY m . □ 

So then we have 



w = w 
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Lemma 7.7. We can write F° ^ as a linear combination of {T u -i$i }wew M as 

^ = Vol(I M )- 1 E ^NV*?*'*- (24) 
t«eWM 

Proof. Since {Ta-iWi^}^^ is the basis of Ind^, (£, VO^S we assume 



uiew M 

for some 6 W G C. Let T^m^ acts on both sides of the equation and take the value at 
w^f, we obtain that 

1 = 6 w (c w mK)) _1 Vo1(Im), 
completing our proof. □ 

Next we consider the action i2(Chy aChj ) on F° ^ for a G T^. We have 

Propostion 7.8. 

i?(Ch lM aCh lM )F^ = E " K)^» ■ T„-x^ 

Proof. Consider i^Ch^aChyJ^. Note that it belongs to Ind^ (f, ^>) Im , so by 
lemma (17.41) . we only need to consider its value on Wm- Note that when a G T^, we 
have the decomposition 

TmoTm = N M ' 1 X°aB^ I j. 
Since ^{ is iM-invariant, and Vol(X°) = Vol(B^j) = 1, so 

Jn m x x° 

Suppose it is non-zero, then by considering the support of ypf'^, 
so 

wNl M aXw™ n B°, jWo M N^Y° + 

Note that 

wN^ M aXw^ C BmwwJ'Nm x J 

and 

BmJ^N^Y C B m <N m x J. 
So by Bruhat decomposition of M we have 

w = e. 
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This implies that R(luoIu)^f^ is propotional to V&f . Consider R(IuqIm)^i ( e )j 
which is equal to 

Note that tyf^ £ Indg the integral is equal to 

£5£ (a) [ dndxtyf^ia^nxa). 

Considering the support of \&i , 

£51 7 (a) / dn*^(a- 1 nxa) = ^1 7 (aWolfNj/X n aNr^XV 1 ) 

When a £ T^, aN-p^a" 1 £ N'^X , so 

Vo^N^X n aN^ M X°a _1 ) 
=Yol(aN^ u X°a- 1 ) 



So we have 



=^ j (a)Vol(I M : 



R(T M aT M )^ = £V>)Vol(I M )*P. 



Applying this to both sides of equation f[!M|) . our proposition is proved. □ 

8. 7-FACTOR 

In this section we assume (x, £) is generic. We are going to show that 

Theorem 8.1. Let x = £ = r(X; £) fre a function on 

(X, 9^en by 

n 

r(x,£) = II r 1 (x«-X6 + i)C" 1 (xa + X6 + i)-Ilr 1 (xi + i) 

l<a<ft<n j=l 

m 

■ n r 1 (e.-6+i)r 1 (e.+6+i)-n( 1 +€i(p)W i ) 

l<a<b<n j=l 

m (n-m)+j-l >/ _ t i JA n m i i 

n n ;}* , % A • n n cfe + & +ha-xi+ & + 

j=l i=l Ai "T "I" 2 J i=l j=l Z Z 

T/ien /or any fixed g £ G, 

^x,€,V'(- R (^) F x' F °^) 

r(x,0 

is Wg x WM-invariant. 
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If we can prove that for (w, w') = (w a , 1) and (w, w') = (1, wp) where a is a simple 
root in G and (3 is a simple root in M, 

T(wx,w'£) = l wx , w i U ,(R(g)F° wx ,F° w ,^) 
r( X ,0 l X M{R{g)F° x ,F°M) 

then theorem (18. ip is implied. Since we can calculate the left hand side above directly, 
we only need to consider the ratio 

>"UL>X,W'£,'*P 

W(i2fo)F°,F^) ' 1 J 

We obtain its value by the uniqueness of the pair l x ,^- To be precise, for (w,w') G 
Wq x Wm, let 

Then l x ^ w y is also a pairing on Indg G (x) <8> Indg^ (£, satisfying Condition A. 
By the uniqueness of such pairing, there exists a constant, which we denote by 
7(x, £, w^, w') , such that 

= 7(X> 6> w')l x ^. 
Then, since T*F X = c w (x)F° x and T^rFj^ = c^^F",^, we have 

(^)F° X ,F 



Z x ^(fl(^)F0 > F^) 
=c; 1 (x)c;- 1 (07(x^,«',«;') 

So in the rest of this section we calculate j(x, £, w a , 1) and 7(x, £, 1> ^/j)- The result 
will be stated in theorem (18.31) and (18. 7|) . which implies the theorem (18.11) . 

8.1. The calculation of j(x, £, w a , 1). 

Let I m j = Im x (X^Z ), and let 

F l*l> = F C^( Cn i M j)- 
. For w G W G , let & x>w = F x (C1i Igu;Ig ). Then by theorem 3.4 in [2] 

So we have 

i(y£w d-c m ■ l ^AR(^) ° (^ aX ,i + g^Ojy f26 , 

i x ,^(-R(A^) o (<£ Xil + $ X)t0 J, F^) 
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Let i' — i — (n — m). The result of the calculation is stated as the proposition 
below: 

Propostion 8.2. For generic (x, £), the value o// X) ^^(i?(Awp) o ($ Xi i + $ x >Wa ), F^ ^) 
equals 

Vol(I G )Vol(I MJ )|pr 1 (l - (XiX^i)(p)lP\) 
for a = Ci — ej + i, 1 < i < n — m — 1, and equals 

Vol(I G )Vol(I MJ ) • (ipr 1 - 1) i HP I torn) (P) 

(i-(x^ 1 )(P)iPi^)(i-(xr + 1 1 e, + i)(P)iPi^) 

/or a = &i — e{ + i,n — m < i < n — 1, and equals 

voi(i G )voi(i M oiprMi-xn(p)ipi) 

for a = 2e n . 

Substituting this in (1261) . we have 
Theorem 8.3. For generic (x,£)> ^ e 7 -factor j(x, w a, 1) is equal to 

M C(xi - + 1) 
a[x) C(xw-» + i) 

for a = Ci — ej + i, 1 < i < n — m — 1, and 



c a {x) 



({Xi - x l+ i + 1) C(x*+i - fr+i + i)C(~Xi + + |) 



C(x<+i - Xi + !) C(Xi - fr+i + |)C(-Xt+i + Ci'+i + 1) 

/or a = — ej + i, n — m < i < n — 1, and 

c (X) ■ CiXn + 1} 

for a = 2e n . 

Recall that for (x, £) G 2 C , we have 

First we calculate I X) ^(Chi G , Chi j)(^ w o), in fact, 
Lemma 8.4. For (x>£) G we /iave 

WCh lG , Ch lMj )(A W o) = Vol(I G )Vol(I M ). (27) 

Proof. By definition, 

I Xi ^^(Chi G , Chi 3 )(^ w o) = / JC^^xwoX^x^dxdx' 
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From now on we write K, x ^ by K, for simplicity. Note that 

A" 1 = d m (-l, -1, -l)Ad(-l, -1, -1) G T° AT° , 

so we have 



dxdx K,{xwq\ l x') = \ dxdx' JC(xwoXx') . 

!gxI m J -/IgxImJ 

The proposition is implied if lC(xw Xx') = 1 for all i G I G and x' G Imj- To show 
this we note that Ig = BqA^q' 1 and Imj = X 1 A^' 1 B M Y°Z . So by the definition of 
/C, we only need to show 

K.(n'w Xxn~) = 1 

for n! G A^q' 1 , x G X 1 and n~ G A^' . Note that when x G X 1 , we have Ax G T M AT M , 
and note that T M normalizes both Nq 1 and N^' , so it reduces to show that 

K,(n'woXn~) = 1 

for n' G A^q' 1 , and ri~ G N^' 1 . The proof for this is similar to that in lemma (18. 100 . 
so we just skip it here. □ 

Now we consider the calculation of I X) ^(Chi GlUa i G , Chi Mj ) (Xwq) . First, by a similar 
method as in lemma (18.101) . we have 

I x ^ v ,(Chi Gtl)a i G ,Ch lMj )(Ato ) = |p| _1 Vol(I G )Vol(I M ) / o dn a K,(w a n a w X). 

Combining this with (1271) . we have 

I X!?)V ,(Chi G + Ch lGt(;celG ,Ch lMj )(Ai<;o) = Vol(I G )Vol(I M )(l + IpT 1 / dn a JC(w a n a w X)) 
Note that w a n a (t) G T G T Q (t~ 1 )N Q (— t)N„ Q (t~ 1 ), so it is equal to 

Vol(I G )Vol(I M ) • + [p]- 1 J^dt^T^t-^won^r^X)) . 
We consider the calculation of this case by case. 

Case a. When a = — e,- l+ i with 1 < i < n - m - 1 we have n a (t~ 1 )X = An Q (t _1 ) 
if i < n — m — 1, and n a {t~ l )X = An Q (t _1 )u for some u G U with ifjy(u) = 1 if 
% — n — m — 1. So 

KiT^r^w^it^X) = lOT^r^WoXn^r 1 )), 

and so 

dtJCiT^r^won^r^X) = [ dtixix^mt 1 ^- 1 )- 

t\<l J\t\<l 
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We let 

i(n) = f 1 1 dtixix^mrMt- 1 ) 
j\t\=\p\ n 

HxixA(P)T\P\- n / , , Wit- 1 ). 

J \t\=\p\ n 

Using lemma (IS. lip below we have 1(0) = 1 — |p|, 1(1) = (—l)(XiXi+i)(p)\p\i an d 
I(n) = if n > 2. Combining these we have 

l x,tA Ch ia + Ch lGtI)Q i G , Chi Mj )(Aw ) 
=Vol(I G )Vol(I MJ )|pr 1 (l - (y,y7 + i)(P)IpI), 

which is the first part of proposition ( 18.21 ) 

Case b. When a = e, — e^+i with n — m < % < n — 1, we let 

j-th 

tj = (1, 1, 1 + 1, 1) e F m . Then n a (t~ 1 )\ = X(ti) if i = n - m, and 
^a(£ _1 )A = X(tj/ + i)?7, Q ,(t~ 1 ) if z > n — m + 1. Here i' — % — (n — m). Note that 

W(yX.(tj) = d'(tj)lUo\d'(tj), so 

UTJr l )w nJr l )X) = [ dt \t\ x ^'+^\t + ll-Xi+i+fc'+i-l. 

7|t|<i 

To calculate this we apply the lemma 8.6 in [7]. 

Lemma 8.5. Suppose \ and x' are two unramified character on F* , then 

i-|pI 2 (xx')(p) 



1 + lPl- 1 / dt x (t)x'(i + t) = (\p\- 1 -i) 

Jo 



2 



o W V 7 V ' ' / (1-|P|X(P))(1-|P|X'(P))' 

Applying the lemma for x — ' anc ^ X' = Xi+i£t'+i| ' l - ^) we have, when 

a = Ci — Ci + \ with n — m < i < n — 1, 

I x ,^(Chi G + Chi GWalG , Chi Mj )(Aw ) 
=Vol(I G )Vol(I MJ )(|p|- 1 - 1) l-|Pl(XiX^)(p) 

(i-(x^i)(P)|p|^)(i-fe 1 i^ + i)(P)|p| 

which is the second part of proposition (18.21) . 
Case c. When a = 2e n , we have 

So 

dHC(T a (t~ 1 )w n a (r 1 )X)= I dtXnitM- 1 ^- 1 ). 
t\<i J\t\<i 
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Similar to Case a, if we let 

-i„/./+-i'* 



J(l) = / dtXnW|t|"V(*" 



Jltl=lpl» 

then by lemma ( 18. lip . 



(1-lPl if «' = 0; 
fy) = -Xn(p)|p| if « = 1; 
[o ifz>2. 

So 

W^Ck-ic + ChiQ^ig, Chi Mj )(Aw ) 
=Vol(I G )Vol(I M . I )|p|- 1 -(l-X„(p)|p|), 

which is part 3 of proposition (18.21) . 

So by the calculation in the Case a, b and c, we have proved proposition (18. 2p . 
which implies theorem 



8.2. Calculation of j(x, £, 1, Wp). 
Let Xi — Xn-m+i for 1 < i < m. Let <? Xi i = F x (Ch(I G )) as before. Recall that 

Im = Im x J°- Let 

®Z,tP,w = F aA C h M wl M )- 
Similar to theorem 3.4 in [2J, we have 

) =^(0(^^,1 + ^,^)- ( 28 ) 

So we have 

7(x,£, i, »,) - ■ WW)(il 5— + » ttJ (»> 

What we are going to show is 

Propostion 8.6. For generic (%,£); ^ e of 1 X! ^(R(Xwq)$ Xi i, ^^l + ^^wp) 
equals 

ipr^i-iPirvoK/^voK/M) n c^-^+o) 

C(xi - Xi+i + i)Cte - &+i + 1) 
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when (3 = e, — e i+ i, and equals 

TO — 1 1 

\p\-\l - |p|) m Vol(J G )Vol(J M ) J] ((-Xj + ti + 5) 



(1 Xm 


H)(p)(i + £to 




Hp)) 


(1 ~~ Xm^ml 




^(P))(l -XmCrr 


1 1 


*(p)) 



w/ien /3 = 2e m . 

Substituting this in (T2§1) we have 
Theorem 8.7. For generic (%,£), #ie 7 -factor '"fix,!;, l,uip) equals 



C(6 - 6+1 + i)C(-Xi + 6+1 + |)C(xi -6 + 1) 



C(-& + 6+1 + i)C(xi - 6+i + \)C{-Xi + 6 + 1) 

when (3 = e, — e i+ i, and equals 



C(-6n - Xto + |)C(-^m + Xm + |)C(2£m + l)C(-£m + |) 



C(6n - Xto + |)C(^m + Xm + |)C(-2^m + l)C(Cm + |) 

when (3 = 2e m . 

To prove 18.6} first we have 
Propostion 8.8. For (%, £) G 

1 - IpI 



I x ,^(Ch lG , Ch lM )(A<) = Vo1(Ig)Vo1(I m ) II 



ii-xj^MlPl* 

Proof. By definition, we have 

I Xi ^Ch lG ,Ch lM )(AO = / / dadm/CQ^A^m). 

Jin •'Iivr 



Since A G Im, and K is left B G -invariant and right B^-invariant, and Vol(B G ) = 
Vol(Bgj) = 1, we have 

/ /_ dg dm/C(gu>?A _1 m) = / dn dn' dx'K{nw Q n'x'). 

J la JT m Jn-' 1 Jn^xo 

By a similar discussion as in lemma fl8.10p . which is given later, we have, for all 

1 

J dx'JC(nw^n'x') = J q dx'K{w^x'). 
I x ,^(Ch lG ,Ch lM )(Au; G ) = Vol(I G )Vol(I M ) dx'K{w$x'). 



n G Nq' 1 and n' G N^ 1 , 



So 
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Parametrize X° as X° = {X(x) = X(xi, ...,x m )\xi G O for all i}, then, 
WqX(xi, ...,x m ) = d m (xi, ...,x m )wQ\d m (xi, ...,x m ) if all Xj are non-zero, and 
JC(wqx') = otherwise. So 



n 



i-ipi 



where the last equality follows from the following lemma. □ 
Lemma 8.9. Let \ be an unramified character on F* . Then 

We skip its proof. 

Next we consider I x ^^(Chi G , Chj MW/3 j M ) (Xwq) . By definition, it is equal to 



dx _ _ 6lx'}C(xwqX 1 x'). 

Jin JI-mwbIm 



Ig •'Im'^Im 



Note that A G I m , and Iuw^m = N ' aW^N^ /3 J°B^ [ . So the above integral is equal to 



n g -^m Jn °° Jj0 



i dn l dn' / dn.-^ / dn K,(nw n'wpn-pu) 



We claim that 

Lemma 8.10. For any n G Nq' 1 and n' G N^' 1 , 

dn_,y jo d^KV W ) = i dn_, ^ duK( W - W ^u). 

Proof. First we have 



N ^ J jo JN^p J jo 



dn-/3 J ^ du JC(nwQn'w/3n_pu) = J q dn.^ y dufc{nw^wpn-pu). (30) 

for some n G Nq' 1 . This is true because 

(1) t« %\ G = Njj, 

(2) N^N^ c B° m N g '\ 

(3) /C is left B^-invariant. 

Continue the calculation, the right hand side of equation ( l30i) is equal to 

dn_^ J ^duK{wQnwpn-pu) (31) 
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for some n G Nq. We write n as n = n^J^, y, z)n-s where n-y G Ng, G N^ «, 
J(x, y, z) G J 1 , and n 3 G U 1 . Here N M ^ is the subgroup of N M generated by all 



Our lemma will be proved if we remove n\n^{x, y, z)n 3 in the integral, which is by 
the following steps 

(1) First, tt-3 can be removed because U 1 is normalized by wp, N°^, and J°, and 
/C is right U 1 -invariant. 

(2) Note that J 1 is normalized by wpN^p, we can remove J 1 (x, y, z) by a change 
of variable in the integral of J°. 

(3) We can remove n<i because Nj^ % is normalized by wpH°_ p, and Nj^ * normalizes 

J°, and JC is right ^-invariant. 

(4) Finally, note that Npwp = wpN^_p, we can remove ri\ by a change of variable 
in the integral of N°_ * 

So the lemma is proved. □ 

By the lemma we have 




WChio, Ch lM ^ lM )(A M ; G ) = |p|- 1 Vol(I G )Vol(I M ) f dt f dxJC(w$wpn-p(t)x) 



Combining this with proposition (I8.8p . we have 



(Ch lG ,Ch lM + Ch IWM )(A<) 
=Vol(Io)Vol(I M ) (fl - }~ lP \, ? + IPl" 1 / dt f dxlC(w%wpn_p(t)x) 




Now we calculate 




Note that wpri-p(t) = n-p(—t 1 )N i a(t)T ( g(t), so 
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Below we discuss this integral case by case. 
Case a. When f3 = — e i+ \. 

We parametrize X° as X(x) = X(xi, ...,x m ) with Xj G O. Then we have 

X(xi, X m ) ^ ^ X(xi, Xj_i, t Xj, Xi -\- tXi+i, X{J r 2i • • • , %m) 

and 

(^)(i>(t)) = (6tf 1 i-r i )(t). 

Let x(z, t) = (xi, -Xj + tx i+ i, x i+2 , ...,x m ), then 
t<i dt jf o dx/C(^ G a;^ {t)T ' 3(i) )(^^^(r /3 (t)) 



dt / dx, (e.^witr^cti/fxcxCt.t))). 

Note that if none of the components of x(i, t) is zero, then 

w^X(x(i,t)) = d m (x(i,t))w£\d m (x(i,t)) 
and /C(u>JfX(x(i, t))) = if otherwise. So the integral above is equal to 

11 / (xj 1 ^ j \-\^)^ j )dx r 



j£{l,2,...,i-l,i+2,...,m} 



/ dtdXidXi+i^i^Al ■ I ^(t) 

J|t|<l,|xi|<l,|a! i+ i|<l 

(x* rl &i ■ r^)(t _l xi) (xr + \Ci+ii • r')(-zi+tei+i) 



By lemma (18.91) . it is equal to 

n 



je{l,2,...,i-l,i+2,.,m} 1 ~ Xj ^'(P)IPI 5 

/ dt dx t dx i+ i I • I -1 ) (*) 

J|t|<l,|xi|<l,|x i+ i|<l 

(xi^&l " r^)(* _1 «i) (xii&fil ■ r^)(-£; + tai+i) 



(33) 



Let I(a, 6) be part of the integral above for \t\ = \p\ a and \xi\ = \p\ b with a,b > 0, 
then the integral is equal to J2 a ,b>o I( a ; When a > b. i.e., |xj| > we have 

| X^ ~t~ tx^-j-i I — — I Xi J . So 

I(a,6) = (l-|p|) 2 |p|fxr b xr + \te +6 - 
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When a < b, i.e., \xi\ < \t\ we can change the variable x i+ i — > x i+ i + ^, we have 



i( a ,b) = E(i - \p\) 3 \p\^xr h x^r c ck!+i, 

c>0 



where the summand for c corresponds to = |p| c after the change of variable. 

So applying this to equation ( 1321) and by some calculation, 



I x ,^(Ch lG , Ch lM + Ch lM ^ lM )(A<) = Vol(I G )Vol(I M ) 

rir^77^ + |prl n : ^ii Ei(M) 
vi= i i - Xj ^-(p)IpI 2 1 - Xj ^-(p)IpI 2 a,6> 

=i P r i (i-ipirvoi(/ G )voi(/ M ) n a\-xj+Q 

C(& -Xi + iKfe+i - Xi+i + |Kte - + |)C(— + ^ + i 
C(x< - + i)C(6 - 6+i + 1) 

which is the first part of proposition ( 18.61) 
Case b. When (3 = 2e m , we have 



(6^x^(0)= u*)i*r': 



and 
So 



X( a ; 1 ,...,x m )^W T ^*)=X(x 1 ,..,x m _ 1) t- 1 x m )Y 1 (-x m )Z(t- 1 x^). 



dt / dxK(w^x n ^ t)T ^)(^)(T8(t)) 
t\<i Jx a 

[ dt [ dx i ^(t- 1 xl)^ m (t)\t\-ljC(w^X(x u ...,x m . 1 ,t- 1 x m )) (34) 

J\t\<l JXiGO 



i - IpI 



= II : -i 'i',^ (35) 

J= i • I 2 (P) 

• / dtdx m (r m l u\ ■ \^){t- l x m ){u\ ■ \-h(t)^(t- i x 2 m ). (36) 

Jt,x m eo 

Let l(a,b) be part of the integral above when \t\ = \p\ a and \x m \ = \p\ b . To calculate 
I(a, 6), we apply the following lemma which can be proved by direct calculation 

Lemma 8.11. Suppose \x\ = \p\ l , then 

'IPP'(I-IPI) ifj<i 

^{t~ x x)dt = < 



*I = |P|- 



-|p| i+2 ifj = i+l 

ifj>i + 2 
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SoI(a,6) = (l-|p|) 2 xr 6 eLlPl^whenO<a<26, and I(a, 6) = _(l-|p|)^|p|^ +1 
when a = 26 + 1, and I(a, 6) = when a > 26 + 2. Applying these to equation 
and by some calculation, we have 



I x ,^(Ch lG ,Ch lM + Ch lM ^ lM )(A 



w 



TO — 1 1 

\p\-\i - | P |) m voi(/ G )voi(/ M ) n c{-xi + d + 5 : 



(1 - Xto 


l-l)(p)(i + e m 




Hp)) 


(1 — Xm^ml 




^(P))(l-Xm&r 


!, | 





which is the second part of the proposition (18. 6ft . Combining Case a and b, we have 
proposition 18.61 which implies theorem 18.71 



9. Formula for generic (x>0 

In this section we discuss the value of I°^(p d Ap f ) for generic (%, £) where d G A+ 
and f G A+. First we claim that 

Lemma 9.1. For any g G I G and m G Im, we have 
in the double coset f/K M j\G/K G . 

Proof. When d G A+ and f G A+, we have p d B^p~ d G and p f B G p~ f G B G , so 
it suffices to prove the lemma when g G Nq' 1 and m G N^' 1 , which is true by the 
method in lemma ( 18.101) . □ 

Now we consider 

r- 1 ^) • / Xi5 , ( J R(A Wo G ) J R(I G p^I G )F;, J R(T M p" d T M )F^). 

By lemma (19.11) it is equal to 

r- 1 ( x ,ovoi(i G )Voi(i M ) J xo dxi^(pW). 

On the other hand, by lemma (17. 3p and (17. 8p . it is equal to 

r- 1 (x, E c w o (w X ) (wxs~l ) (p- f )c w u (w'O (w'ts'l ) ( P - d ) 

weW G ,w'eW M 
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So we have 



E ( S? ^ ( w * 5 *c ) (P" f ) c < KO Ktf^ ) (P~ d ) (37) 



To calculate the right hand side of the equation above, we use the following lemma 
without proof. 

Lemma 9.2. For generic (x, £), and (d, f) G A+ x A+, /ei 

B(x,e,d,f) = £ ^(x,£,^0(^x)(P f )(^O(P d )- 

Suppose £>(x, £, d, f ) is Wq x WM-invariant for all (d, f ) £ A+ x A+, then we have 

A(x, w, w') = A(wx, w'£, e, e). 

By this lemma, the summation on the right hand side of (13 7p is determined by its 
summand at (w, w') = (e,e), which, by proposition 18.81 is equal to 

^G(x)(x5Bl)(p- f )c % M(0(^ B i)(p- d ) • W^o)*?, ■ r-\ x ,0 
=(i - |p|) m b( x ,0d(x)d / (0(x5B!)(p-0(^Bi)(p- d )Voi(i G )Voi(i M ), 



where 



and 



d(x)= IT C(Xa-Xb)({Xa + Xb)l[({Xi), 

l<a<b<n i=l 

m 

d'(0 = n c(£ a - &)c(£« + 6) n c(2^), 

l<a<b<m j'=l 



Mx,o= n r 1 (xi-e i + ^)- n r 1 (-x 4 + e J + ^: 

i<j+n—m i>j+n—m 

n c^+b- n cWe,^) 

l<j<m Z l<i<n Z 

l<j<m 



Applying the lemma 19.21 we have 

/ xo dxi^(pW) = (i-|pirr( x ,o 



£ b(wx,wX)d(w X )d\wX)(w X S^)(p- f )(w'^)(p^ 

togWctt'eWM 



(3? 
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Though this is not a direct formula for 1° ^ ^,(p d Ap f ), we have the following theorem. 
Theorem 9.3. Let /(d, f ) = l£^(p d Ap f ), and let 

L(d,f) = | xo dxI° x ^(pW), (39) 
then /(0,f) = L(0,f). Moreover, there exists a(d') > such that 

l(d, f) = Y. a(d')L(d', f + d - d') 

d' 

where the sum is over the set {d' | d' < d, d' £ A+, f + d — d' E A+} ; and we have 
a(d) > 0. 

Proof. We have 1(0, f) = L(0, f) by the K M j-invariance on the left. Because of this, 
if we can prove 

L(d,f) = ^6(d')/(d / ,f + d-d') (40) 

d' 

where d' runs over {d' | d' < d, d' e A+, f + d- d'GA+} with b(d') > and 6(d) > 
0, then the theorem is implied. Expand the right hand side of equation (I3TJ1) . and then 
by lemma (16.31) . L(d, f) is a linear combination of 1° ^ ^,(p d X(c)p f ) with non-negative 
coefficients where < c < d. Note that 1° ^(p d X(c)p f ) = 1° ^(p d ~ c Ap f+c ), and 
that (d — c; f + c) satisfies the assumption in lemma (16. 6p . So by lemma (16. 6p . 
I°^ ^(p d ~ c Ap f+c ) = 1° ^(p d 'Ap f+d ~ d ') where d' satisfies the conditions in our theo- 
rem, so we have (HOI . To show that 6(d) ^ 0, note that when x e (0*) m , 
1° ^(p d X(x)p f ) = l(d, f), so 6(d) > (1 - |p|) m > 0, completing our proof. □ 

10. Normalization 

By equation (1381) . we have 

I°^(e) = (1 - |p|)"T(x, ■ E M«% w'0d(w X )d'(w'0. 

i»ew G ,ra'ew M 

Now we calculate the value of 1° ^(e). The method is similar to that in section 11 
in [7j. Our conclusion is 

Theorem 10.1. The value o/I° ^ at identity is given by 

m 

?^(e) = (i-i*>irr(x,oc(irnr 1 (20- 

Let C = X^eWcw'eWM M^Xj w '£)d-(wx)d'(w'£), then what we need to show is 

in 

c = car new 

i=l 
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In this section we simply denote Xi(p) ( re sp. £j(p)) as Xi (resp. £_,•) for convenience. 
Let bx(x) be a function defined on % £ C™ and b 2 (£) a function on £ £ C m , we define 

-4w G (t>i(x))= sgn(w)hi(wx)- 

uigW G 

and 

-4w M (b 2 (0) = £ s#n(u/)b 2 («/£)• 

Then it is not hard to see that 

Lemma 10.2. For w £ Wg and w' £ Wm, we /iave 

^w G (bi(wx)) = s 9n(w)Aw G (h 1 (x)), 
Aw M (b 2 (w'£,)) = sgn(w')A WM (b 2 (£)). 

For e £ Z™ and /i £ Z m , let X e = UiXT and £^ = • We say e (resp. fi) is 

regular if we = e (resp. w'/x = fi) implies w = e (resp. w' — e). Then we have 

Lemma 10.3. For non-regular e and fi, 

A WG ( X e )=0; A Wm (^) = 0. 

Let pi — (n — ~, n — §, |) and let p 2 = (m, m — 1, 1). Let 

P(x) = x pi ; Q(0 = £ P2 - 

Note that p% is the half sum of postive roots in G = S02 n +i(C), and p 2 is that of 
M = S"P 2m , and note that Wg = W G , so, by the Weyl character formula on G and 
on M, we have 

( 1 \n I 1 \m 



P(x)^w G (P(x)) Q(0^w M (Q(0) 

From which we know that 



p(wx)^ Wq (p(x))' QK0^w M (Q(0)' 

So we have 

r = / ir+ n. (^wc °^w M )(b(x,QP(x- 1 )Q(r 1 )) 

1 ] ^w G (P(x)Mw M (Q(0) 
Note that sgn(w^)sgn(w^) = (-l) m+n , iu£(x) = X" 1 and wf(£) = So 

Av G (P(x)Mw M (Q(0) 
To calculate C, we need to simplify (*4.\y G ° ^w M )(b(x \ £ _1 )P(x)Q(0)- 



(41) 
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Lemma 10.4. Let 

A(Xl ° = rEr nr=i c-K-xi + + IX 31 ^ -0 + 1 



b(x- i ,r 1 )p(x)Q(e) 



Mw g °^Wm)(^(X,0) 

^w g (P(x)Mw m (Q(0)' 
Proof. Consider b(x _1 , £ -1 )P(x)Q(£), wihch is equal to 

n c^-xi+tj+h- n c-^+xi - 6- + ^) 

i<j'+n— m i>j+n— m 

n r i (-o+^)- n cH-xi-Zj+l)-**?*. 

l<j<m l<i<n 
l<j<m 

Let b(x" 1 ,r 1 )P(x)Q(0 = E^c e ^ ■ then 

e,/i regular 

By considering the power of Xi f° r those (e, fi) with c fiM 7^ 0, we have 



\ + {n — i) — 2m, | + (n — i)] when 1 < i < n — m 
[| — m, m — |] when n — m + l<z<n 



Suppose e is regular, then for n — m + 1 < i < n, {|ej|} must be a permutation 
of {|,|,...,m — |}, which implies e n _ m = | + m, which then implies e n _ m _i = 
I + (m + 1), and so on. Eventually we have, q = | + n — z for 1 < z < n — 
m. In other words, for 1 < % < n — m, should attain its upper bound. This 
implies that (.A\y G "4w M )(b(x _1 j £ _1 )P(x)Q(0) remains the same if we divides 

n^rn^iC-H-Xi + + Dr^-x* - o + 1) from b( X - 1 ,r 1 )p(x)Q(0. which 

equals A(x, £)■ ^ 
Let X = (Xi, ■ • • , Xm) e C m such that Xi = Xn-m+i- Let 

^(x,0 



A(x,e) 



1 A , 2 



=nr 1 (-xi+^ + 5)11^-0 + 5) 

i<j z i>j z 

n r^-o + b n c^-o + b^r, 

where pi = (m— ~, n — |, ... ,|). Then we have the following lemma. 
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Lemma 10.5. If we let P(x) — X^S an d let 

(A Wm o A Wm )(A(x,0) = E sgn(w 1 )sgn(w 2 )A(w 1 x,w 2 C), 

then 

c _ (Aw M oAw M )(Mx,0) 

^lw M (P(x)Mw M (Q(0)' 

Moreover, the value of C is independent 0/ (%,£). 
Proof. What we actually need to prove is 

(^w Q % M )(^,0) = (^w M ° Av m )(A(x,Q) 

^w G (P(x))^w M (Q(e)) ^w M (P(x))^lw M (Q(0) 

and that it is independent of (x, £). Let A(x,£) — X^, M ^e,/xX e £ M - Then by the 
discussion in the previous lemma, 

a — — V n — i, for 1 < i < n — m 
2 ~ ~ 

6j G [— (m — -), m — — ], for % > n — m. 

So for each regular e, if we let e' = (e n _ m+ i, . . . , e n ) G Z m , then 

(1) e = (n - \,n - §, . . . ,m+ |;e'). 

(2) Note that X e = 117=1™ xl^" ' so 

A(x, = E rf ^X e '^ 

(3) e is regular if and only if e' is regular with respect to the action of Wm- So 
when e is regular, {|e n _ m+ i|, . . . , \e n \} is a permutation of {|, . . . , m — |}, and 
hence there exists i^i G Wm, such that e = and e' = u>ip~i. 

By a similar consideration on the power of £, if /i is regular, we have 

. . . , |/i m |} is a permutation of {1, ... , m}. 

So for (e, fi) being regular, we let e = u>ipi (so e' = u>ip~i), and /x = W2P2, where 
u>i,u>2 G Wm, and we let d WuW2 = d t ^. Then 

(A Wg oA Wm )(A( X ,0)= E <M^w g °Av M )(x e ^) 

e,/i regular 

= E ^ 1 ^(^w G o^lw M )((x) ,UlPl (0^ 2 ) 

f l,t02 6M 

=*4 Wg (P(x))Av m (Q(0) E d WuW2 sgn(w 1 )sgn(w 2 ). 

mi ,u)2SM 
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and 

GA Wm °-4wJ(A(x,0) = E ^(^w„% M )(x f 'f) 

e,/i regular 

=A Wm (P(x))A Wm (Q(0) E d w 1 ,w 2 sgn(w 1 )sgn(w 2 ). 

So 

(^W G oiw M )(%Q) 

Aw G (P(x))Aw M (Q(0) 
= E d Wl)W2 sgn(wi)sgn(w 2 ) 

_ (^w M oiw M )(AfcQ) 

A Wm (P(x))Aw m (Q(0)' 
completing our proof since d wum is independent of (%,£). ^ 

So to find the value of C one only needs to find the value of 

Aw m (p(x))Avm(Q(0) 

for a special (x, £)• From now on we let Xi = —(tu + 1 — i) and £j = — (m + | — j), 
then we have the following lemma 

Lemma 10.6. If A(wx,w'^) ^ 0, then w = w 1 = e. 

Proof. We denote X-i — ~X% an d £-j = £j- For any weyl element w, w' G Wm, there 
exists a, a', permutations of the set {±1, ±2, . . . , ±m} such that 

(1) a(- i ) = -a({),a'(-j) = -a'(j). 

(2) = x<n, (w'Oj = 

So w = w' = e if and only if a = a' = id. Now suppose A(wx, ^ 0, since 
C _1 (0) = 0, we a and a' should satisfy the following properties: 

(1) For any i < j, x ai - £ a , (j) ^ \. 

(2) For any i>j,Xa i - £<r>(j) ± ~\- 

(3) For any l<j<m, ^'(j) ^ \- 

(4) For any 1 < ij < m, x^ + £<r'(j) ^ \ 

These four conditions actually imply a — a' — 1. To see this we consider A = 
{Xa(i), 1 < i < m} and B = 1 < j < m}. Note that {|x<r(i)|, • • • , = 

{1, . . . ,m} and {\£ a (i)\, • • • , |&(m)|} = {5, . . . , (m - -)}. By property (3), \ B, 
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which implies — | G B. Then by property (4), 1 £ A. So then —1 G A. Then again 
by property (4), | ^ 5, so then — | G 5. And then by property (4), 2 A, so —2 G A. 
Continuing this process, we will eventually have A = {—m,—(m — 1),...,— 1} and 
B = {— m+ 2> —7TI+!, — |}. So a and a' are actually permutations of {1, 2, m}. 
Note that Xk — 6c = — f , so by property (2), <r(z) 7^ cr'(i) f° r an y * > i> which implies 
that 

e-\k) < {a')-\k) (42) 
when 1 < k < m. On the other hand, since Xk+i — Ck — \i we have 

^(fc + l) > (a')- 1 ^). (43) 
by property (1). Combining equation (|42p and (1431) . we have 

(c7') _1 (m) > a-^m) > {a')~\m - 1) > <7 _1 (m - 1) > ... > (oT'C 1 ) > <r _1 (l), 
which implies that a = a' = id. □ 

By the lemma, 

c A(x,Q 

Aw M (P(x))^w M (Q(0) 
where x% = ~~ i n + 1 — an d £?' = — ( m + | — j)- Note that by the Weyl character 
formula, 

n 

Av M (P(x)) = II C _1 (-x« + x 6 )C _1 (-Xa - xt) II C _1 (-Xi)P(x); 

l<a<b<m i=l 

m 

^ Wm (q(0) = n c\-ta + 6)c _1 (-e« - 6) n r 1 (-26-)Q(e). 

l<a<b<m j=l 

By direct calculation we have 

m 

c=c(irnc- i (2«')- 

i=l 

11. Uniqueness of the Whittaker Shintani function 

In this section we show the following theorem. 

Theorem 11.1. Let be a Whittaker Shintani function on G. Let W(d, f ) = 

W x ,^(p d Ap f ) with d G A+ and f G A+. // W(0, 0) = 0, tfien W(d, f ) = for every 
d G A+ and f G A+. 

Combine this with theorem (16.11) we know that for all (%, £), the space of Whittaker 
Shintani function W<S Xi £ ; ^ is of at most one dimensional. The method we use in the 
proof is from [7] and [9] . First we define an order on A+ x A+ as 
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Definition 11.2. Let w^, w[ be the dominant weights of G and M. For any 
(d', f), (d, f ) G A+ x A+ we write (d, f) > W5 (d', f) if 

(1 ) (wi, f ) > (zu h f) forl<k<n-m 

(2) (wi, f) + (w z '_ (n _ m) , d) > (cc7;, f) + (ro ? _ (n _ m) , d') /or n - m + 1 < I < n 

(3) (n7 n _ m +n, f) + (w[,d) > (tu„_ m+i _i,f) + (ro/,d ; ) for 1 < I < m 

Then we have the following lemma 

Lemma 11.3. Suppose (d, f ) G A+ x A+. 

fijJ/(d' ) f')eA+xA+ satisfies 

K MJ p d K G p f K G n ZUK M .,p d 'Ap f 'K G ± 0, 

then (d,f) > W5 (d',f). 
(%) IfuEU and z G Z satisfies 

K MJ p d K G p f K G n ^K M .,p d Ap f K G ^ 0, 

i/ien ipu(u) = 1 and V'(^) = 1- 

Before proving the lemma ( 111.31) . we first show it implies theorem f lll.il) . 

Proof of theorem [11. Consider JK Mj p d K G p f K G ^-Q^x^id) • On tne one nan d ^ is 
equal to 

^(Ch KGpf K G )^(Ch KMjpdKMj )>v(o,o). 

On the other hand, by lemma (111. 3D and themrem (16.11) it is equal to 

£ C d /,rW(d',f). 

(d',f')<ws(d,f),(d',f')eA+xA+ 

Where Cd,f is positive by the second part of the lemma (111.31) . So if W(0, 0) = 0, 
we have 

£ cVf'W(d',f) = o 

(d',f')<ws(d,f),(d',f')6A+xA+ 

for all (d, f ) G A+ x A+. Taking the induction on (d, f) by the order <ws we have 

W(d,f) = 

for all (d, f ) G A+ x A+, completing our proof. □ 

So in the rest of this section we only need to prove lemma (111.31) . To prove the 
first part of lemma All .3j) . we need the following lemma. 
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Lemma 11.4. LetJ\f 2n = {1,2,..., 2n}, and letg, g 1 , g 2 , g 3 E G. For I = 

J = (ji, • • • , 3k) e (N 2n ) k , we denote fi,j(g) = Us=i 9i s ,j s , and 

k 

Aj.jO) = det(g ItJ ) = sgn(a) JJ g is ,j <s y 

a-es k 8=1 

If g — g 1 g 2 g 3 , then we have 
Proof. Since g = g 1 g 2 g 3 , we have 

El 2 3 
9i,a9a,b9b,ji 



a.b 

where a, b runs over Af 2 k n - So 

k 

&i,j(g) = E s 9n(<r) E II 9l,a 3 9i,b a 9b s ,j ais 

v£S k ( ai ,...,a k ),(bi,...,b k )e(M 2 n) k s=1 

Note that Sk acts on (A/2 n ) fc . If we define c s = b a -i^, then 

k 

e n 

9i s ,a s 9a a ,b s 9b s ,i CT fs1 

(oi l ... ) o fc ),(6i,... ) 6 fc )eCA/'2n)* 5=1 
(a 1 ,...,a fc ),(ci,...,c fc )e(Ar 2n ) fc «=1 

Note that for any a £ Sk, 

k k 
IT 9c a t B )i3a(8) ~ n S'cajs- 



So (j4"BJ) is equal to 



<x(s) 



E IT (.'// .-.'/;'' , • E s # n (°") IT 9 2 as , 

(a 1 ,...,a k ),(c 1 ,...,c k )e(Af 2 n) k s=l ^S k s=l 

fc 

E IT {9i s ,a s 9l s ,j B ) ■ A( aiv .., afe ),( Cl ,..., Cfc )(^ 2 ). 

(ai,...,a fc ),(ci,...,c fc )e(A/2 n ) fc 5=1 

which is the formula we want to prove. 

By this lemma we can prove the first part of lemma (111.31) . 
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Proof of first part of lemma M1.3\) . Suppose 

K MJ p d K G p f K G n ZUK MJ p d V'K G ^ 0, 

then p f Wq Xp~ d ' uz = k\p~ f k2p~ d k for some fci, &2 G K G and /c G K m .j. Apply ck; and 
Pi, which are defined as (ITST) and fTTTT) on page M to both sides of the equation. When 
I > n — m + 1, we have 

^Mp f '^ G Ap- d '^)) = -<^,f> - ( W ;_ (n _ m) ,d'). (47) 

On the other hand, note that 

«j(») = Aj 1j j 1 (^). (48) 

where I; = (2n + 1 - I, 2n + 1 - (I - 1), . . . , 2n), and J t = (1, 2, . . . , I). By $0$, we 
have 

«^iP~ f ^P~ d £0 = E //,A(A;i)A AiC (p" f A; 2 p- d )/ c , J (A ; ). 

A,Ce(AT 2ll )' 

Note that for fi^ih) ^A,c(p^hp~ d )fc,j(k) ^ 0, both A and C should contain 
distinct coordinates. Moreover, note that k G K M .i, so fc,j(k) ^ implies Cj = j for 
a\\ 1 < j < n — m. Under these two restrictions it is not hard to see that 

v(A AC (p^k 2 p~ d )) > -Kf) - (wl {n _ m) ,d). 

So 

via^hp-'hp^k)) > ~{w h f) - (ts7,'_ (n _ m) ,d). (49) 
Comparing (1471) and (T4"9"|) we have 

for all n — m + 1 < / < n. Similarly, if we apply ai for 1 < I < n — m or fli for 
1 < I < m, we will have 

(w h f) < (w l} f) 

for all 1 < / < n — m and 

(w n - m +i-i, f ) + (^(, d') < (n7 m _ m+i _i,f) + (w z ',d) 
for all 1 < / < m. So we have the first part of the lemma flll.3p . □ 

Next we prove the second part of the lemma fill. 31) . 
Proof of second part of the lemma M1.3\) . Suppose 

p<Vp f = zukp d \p f k 2 (50) 
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for some u G U, z G Z, k G K m j and ki, k 2 G K G . We need to show that ij)(z) = 
ipu(u) = 1. Consider the element r 2n = (0, 0, . . . , 0, 1) G F 2n . Multiplying r 2n from 
the left to both sides of (j50p . we have 

Here k^, k 2 n are the 2n-th row of k 1 and k 2 . Suppose k^ n = (k\ nl , . . . , k\ n2n ), then 
k 2 

K 2n 



7t 1 /i+/ii.l h /i+/2i,l h /i+/i.i,l n/i^/^i* 1 n^ 1- / 2 ^ 1 i- 1 

"If ^n.l'f ^271,2' •••) V ™2n,n>V "^n.n+l > • • • ' V ^2n,2n-l> ^2 



v 2n,2n) 



(51) 

Note that when k 2 G Kq, each row of it is primitive, that is, it belongs to 2n , 
but not (pO) 2n . So suppose fi = f 2 = ■ ■ • fk > fk+i for some k, then 

by (l5lj) . at least one element in {k\ n 2n _ k+l) . . . , k\ n 2n } belongs to O* . Let's say 
it is ki 



i„„-i 



2n,2n-i+l- 



Let w be an weyl element of G transposing 1 and i, then k w 



has element in O* at the (2n, In) position. Then, by the Bruhat decomposition of 
K (mod p), there exists xx,x 2 ,y±,y 2 G C n_1 , and zi,z 2 G O, such that 



fc 1 ^ = E 1 (x 1 ,y 1 ,z 1 ) k! E x (x 2 ,y 2 ,z 2 ) w * 



where e 6 0*, fc' 6 K,gp 2n _ 2 , an d 

Ei(x,y,z) 



(I x y z \ 

l2n-2 _t x 



\ 



1 / 



So (I5"0|) becomes 

£i(xi,!/i,*i)» d p d 



ft 



\ 



k> p f p- f £i(z 2j y 2 , z 2 r° G wp f = uzfcp d Ap f fc 2 . 



By the definition of w, it commutes with p f , so p f E 1 (x 2 ,y 2 , z 2 ) w o wp f G K G . So we 
just need to show that 



E 1 (x 1 ,y 1 , Zi y d I p d I k> ] p f ] = uzkp d \p f k 2 



(52) 



implies ip(z) = ip\j(u) = 1. We prove this by induction on n — m. 
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When n — m — 1, U is trivial, and Ei(x,y,z) = J(x,y,z). By (1521) . we have 
p f £; 2 p~ f G M J . So fc 2 G K m j. Suppose /c 2 = n x k" where n a G J° and k" G K M , and 
suppose f = (/ 2 , • • • , fn), then we have 

J(zi, Vi, ^i) pd |p d ( fc' j P f j = **p d An?V ft" 

Note that now both sides belongs to M J , we write both sides in the form of J x M. 
Then by comparing the J-part of both sides we have 

lixuyuzxf* = z{\n , () k * A . 

f f 

When f G A+, n\ G J°. So we assume \n\ = J(x 3 , y 3 , z 3 ) with x 3 ,y 3 G O n ~ x and 
z 3 G O. Then we have 

z ■ Z(z 3 - z x ) = 3{x x ,y u 0) pd J(-x 3 , -y 3 , 0) fcpd 

When J(si, yi, 0) pd J(-x 3 , -3/3, 0) fepd G Z, we have 

J(-x 3 ,-y 3 ,0) fepd = J(-xi,-yi,0)" d . 

So z ■ Z(z 3 — zi) = x\ -*yi G Z°. Since z±, z 3 G O, so z G Z°, and hence ^(z) = 1, 
completing the proof for n — m = 1 . 

Assume the lemma is true for n — m — r — 1, and suppose now n — m = r > 1. 
Then E\(x\, yi, 2i) p G U. Since yi, £1) G U° and p d stablizes ^>u, we have 

^u(Ei(xi, yi, 2i) p ) = 1. So from (1521) reduces to show that 

p d f fc' j p f = wz£;p d Ap% (53) 

implies = "0u(w) = 1. Let 

(1 * *\ 
g 1 * } = s P2n _ 2 k H 2n _x. 

Then by (155)1 . p%p~ f G G', so k 2 G K G /. Let fc 2 = uk" where u G H 2n -i and 
fc" G Ks p . Then we have 

p d I k' J p f = uzfcp d A • (w) pf p^" 
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Suppose u = u l u 2 where u 1 G %2n-i an d u 2 G Sp 2n _ 2 . Now both sides belongs to 
G'. Write them in the form "H 2n _iSp 2n _ 2 we have 

u i(fi)*W = 1 

and 

(\ 



u 2 zkp d Xp f k" = p d 



\ 



Jp f 



Note that u G U°, f G A+, and zkp d G M J which stablizes ^u> we have 

On the other hand, by assumption of the induction, when 

f 1 \ 

u 2 zkp d Xp f k" = p d k' \ p f , 

V V 

we have ^11(^2) = i>(z) = 1. So ip\j{u) = ^(uiu^) = 1 and ip(z) = 1, completing 
our proof for n — m — r > 1 . □ 

12. The formula for normalized Whittaker-Shintani function 

Let l x ^,tjj be a pairing between Indg G (x) and Ind^f (£, satisfying Condition A 
and 

Since every such pairing produces a normalized Whittaker-Shintani function, so it 
is unique for every (%,£)■ By theorem (110. ip . for those generic (%, £) where r(x, £) 
has no zeroes or poles, such pairing exists. So applying the Bernstein theorem, and 
the corollary in section 1 in [I], formula (138j) can be extended to a regular function 
on (x,0- Now we summerize our result. 

Theorem 12.1. For every (x,0 G C n x C m , the normalzied Whittaker-Shintani 
function is given by 



dxw^(pW)=c(i)- m nc(2 
x 1=1 



1 



Y, 6(wxXOd(«'x)^(to / e)((«'x)" 1 «y')(i» , )((to , e)" 1 * i )(p e 

w&Wg,w'&W m 



for d G A+ and f G A+. // we /ei £(d', f) = / x „ dxW° £ ,,,(P d ^P f ), ^ 



W^(p d Ap f ) = E «(d')^(d', f + d - d'). 
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with a(d') > and a(d) > 0. Here in the summation on d' runs over the set 

{d' | d' gA+, f + d-d'eA+d'< d}. 

In particuler, 

W^(p f ) = £(0,f). 
13. Application 

Using the formula for the Shintani function, we can give an alternative proof of 
the Theorem 6.1 in [9]. We rewrite the theorem as below. 

Theorem 13.1 (Theorem 6.1 in [9], conjectured by T. Shintani). Let G = Sp 2n and 
M = Sp 2m as defined in our paper, and suppose n = m + 1. Let n and a be the 
unramified representation of G(F V ) andM(F v ) respectively. Let 
z-k = (p xi , • • • , p Xm+1 , 1, p~^ m + 1 j . . . ; p~ Xl ) be the Satake parameters of tx and z& = 
(p^ 1 , . . . , p^ m , p~^ m , . . . , p - ^ 1 ) be the Satake parameters of a with respect to if; so that 
a ®uj^ = Indg Let W°^ be the Whittaker- Shintani function as we defined 

in this paper. Then we have 



't 



L(tt, s) 



K,^\ \tr m - i dt= ^ 7 (54) 

glx \ t -iJ L^(a,s + f)C(2s) 

We denote by LHS and RHS the left hand side and right hand side of above 
respectively. First we have 

Lemma 13.2. Recall that p\ = (n — |, n — |, . . . , |), where n = m + 1 now, corre- 
sponding to the half sum of positive roots in S02 n +i = S02 m +3 ■ Then 

|S -4w (p<->) IPl • 

Note that by abusing the notation the I in I + p\ is regarded as (1,0, ... ,0) £ C™. 

Proof. Since all the data are unramified, the integral on the left is actually a sum 
over t = p l with I £ Z. By the discussion of section El the Whittaker-Shintani 
function vanishes unless / > 0. Substituting W° ^ ^ by the formula we developped in 

. ft 



the previous sections, and note that 5^ Q 



hm, = |t| m+1 , we have 



LHS = c" 1 • 

l>0 



\P\ SI ■ E b(w X ,w'Od(wx)d'(w'0((w X r 1 )(p l 
wew G ,w'ew M 
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Here c = C(l) m IlS=i C _1 ( 2 *)- Recall that p 2 = (m, m - 1, 1). Then 

d( x ) = (-ir +1 p~ ix ' pi) ^w G (p {x,pi} )- 
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So 
LHS 



Aw g {P {x ' Pi) )A Wm {P^ P2) ] 



E 

l>0 



lis 



E ( 

w£W G ,w'£W M 



-l) 2m+1 sgn(w)sgn(w')p- {wx > l+pl) p- {w '^b(w X ,w'0 



(55) 



In fact the summation on Wq x Wm is equal to, by a change of variable w i— >■ wWq 



g 



and w' H- w'w^f, 



£ sgn(w)sgn(w')p {wx > l+pl) p {w '^b(-w X , -w'£) 

w£W G ,w'£W M 

We can further simplify this summation by a similar discussion as in Section QUI 
By the definition of b(x, £)> we have 



na-p 



-Xi±Si+|p(J+m+|)xi 



Here A(x, £) is defined right after lemma 10.4. If we let Wi be the subgroup of Wq 
stablizing xii ■ ■ ■ % Xm+i and W2 the subgroup of Wg stablizing Xii an d let Wo be a 
set of representatives in (Wi x W2)\Wq, then the first bracket is invariant under 
Wm x W2, and the second bracket is invariant under Wi. So the summation over 
Wq x Wm is equal to 

m 

]T sgniwowJKwxWo) o - p-xi±&+^(»+m+|)»))] 



sgn(w 2 WM)A(w 2 wox,wuO] 



V)2,IVM 



By Lemma 110.51 the second bracket is equal to 



c- sgn{w 2 w u )p {w2Woxrpl) P {wM ^ p2 \ 

W2,WM 
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where p\ = (0, m— |, m— |, . . . , |). From this it is not hard to see that the summation 
over Wq x Wm in (lo"o"|) is equal to 

(m \ 
i=i / . 

Substituting the formula to equation [55] we obtain our lemma. □ 

Now we can prove Theorem 113.11 

(Proof of Theorem 1 1 3.1 j) . By Weyl's character formula, for the representation of 
S'027v+i(C) whose highest weight is A = (Ai,...,Ajv) G A^, the trace of x — 

diag{x u ...,x N ,l, x^ 1 , . . . , x^ 1 ) is Tjv(A; x) = e (x * N _. + ^ Xi _ (N _ j+ i } — -. The func- 

det(a; i ™ — x. ™ ) 

tion Tat(A; x) is in fact defined for all A G Z N . For any set A = {a\, . . . , on}, we let 
A* (A) = J2scA,\s\=i(Hs&s a s)- Using these notation, we can express LHS as 

LHS = ]T I" 1 )' A ' ( r -) • T m+ i((/ - r, 0, . . . , 0); z T )\p\ ls . (56) 

Z>0,r£{0,l,...,2m} 

Here is the set {£i + |, . . . , £ m , + ~, — £ m + |, . . . , — £ x + |}. Next we consider RHS. 
By the discussion in [21 Theorem 3.1], we have 

a>0 

So by the notation introduced above, we have 

RHS = ]T (-iy A r (r*)f m+1 ((a, 0, . . . , 0); ^)|p| (a+r)s . (57) 

a>0,re{0,l,...,2m} m 

To show that ( I5"B"1) equals (13T1) . note that in (l5"6l) if / < r, then / — r G { — 1, • . . , —2m} 
since < r < 2m. Then it is not hard to see that T m+1 (/ — r, 0, . . . , 0) = by its 
definition. So one can replace the summation from I > to I > r. Then by a change 
of the variable I = a + r with a > we have (1561) equals ( 1571) . □ 

References 

[1] William D. Banks. A corollary to Bernstein's theorem and Whittaker functionals on the meta- 
plectic group. Math. Res. Lett., 5(6):781-790, 1998. 

[2] W. Casselman. The unramified principal series of p-adic groups. I. The spherical function. 
Compositio Math., 40(3):387-406, 1980. 

[3] W. Casselman. Introduction to the theory of admissible representations of p-adic reductive 
groups, http://www.math.ubc.ca/~cass/research.html, 1995. 

[4] W. Casselman and J. Shalika. The unramified principal series of p-adic groups. II. The Whit- 
taker function. Compositio Math., 41(2):207-231, 1980. 



THE WHITTAKER-SHINTANI FUNCTIONS FOR SYMPLECTIC GROUPS 



49 



[5] David Ginzburg, Dihua Jiang, Stephen Rallis, and David Soudry. L-functions for symplectic 
groups using Fourier- Jacobi models. In Arithmetic geometry and automorphic forms, volume 19 
of Adv. Lect. Math. (ALM), pages 183-207. Int. Press, Somerville, MA, 2011. 

[6] David Ginzburg, Stephen Rallis, and David Soudry. L-functions for symplectic groups. Bull. 
Soc. Math. France, 126(2):181-244, 1998. 

[7] Shin-ichi Kato, Atsushi Murase, and Takashi Sugano. Whittaker-Shintani functions for orthog- 
onal groups. Tohoku Math. J. (2), 55(l):l-64, 2003. 

[8] Atsushi Murase. L-functions attached to Jacobi forms of degree n. I. The basic identity. J. 
Reine Angew. Math., 401:122-156, 1989. 

[9] Atsushi Murase and Takashi Sugano. Whittaker-Shintani functions on the symplectic group of 
Fourier- Jacobi type. Compositio Math., 79(3):321-349, 1991. 
[10] Xin Shcn. Local theory for tensor L-function for symplectic groups, preprint, September 2012. 

School of mathematics, University of Minnesota 

520 Vincent Hall, 206 Church st. SE, Minneapolis, MN, 55455, USA 

SHENXl25@UMN.EDU 



